<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.9.4"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Flow-IPC: ipc::shm Namespace Reference</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr id="projectrow">
  <td id="projectalign">
   <div id="projectname">Flow-IPC<span id="projectnumber">&#160;1.0.2</span>
   </div>
   <div id="projectbrief">Flow-IPC project: Full implementation reference.</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.4 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
var searchBox = new SearchBox("searchBox", "search",'Search','.html');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function() {
  initMenu('',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
/* @license-end */
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="namespaceipc.html">ipc</a></li><li class="navelem"><a class="el" href="namespaceipc_1_1shm.html">shm</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="summary">
<a href="#namespaces">Namespaces</a> &#124;
<a href="#nested-classes">Classes</a> &#124;
<a href="#typedef-members">Typedefs</a>  </div>
  <div class="headertitle"><div class="title">ipc::shm Namespace Reference</div></div>
</div><!--header-->
<div class="contents">

<p>Modules for SHared Memory (SHM) support.  
<a href="namespaceipc_1_1shm.html#details">More...</a></p>
<table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a id="namespaces" name="namespaces"></a>
Namespaces</h2></td></tr>
<tr class="memitem:namespaceipc_1_1shm_1_1classic"><td class="memItemLeft" align="right" valign="top">namespace &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespaceipc_1_1shm_1_1classic.html">classic</a></td></tr>
<tr class="memdesc:namespaceipc_1_1shm_1_1classic"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a> sub-module with the SHM-classic SHM-provider. See <a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a> doc header for introduction. <br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:namespaceipc_1_1shm_1_1stl"><td class="memItemLeft" align="right" valign="top">namespace &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespaceipc_1_1shm_1_1stl.html">stl</a></td></tr>
<tr class="memdesc:namespaceipc_1_1shm_1_1stl"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a> sub-module providing integration between STL-compliant components (including containers) and SHared Memory (SHM) providers. <br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table><table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a id="nested-classes" name="nested-classes"></a>
Classes</h2></td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">struct &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="structipc_1_1shm_1_1Arena__to__borrower__allocator__arena.html">Arena_to_borrower_allocator_arena</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight">Implementation of <a class="el" href="namespaceipc_1_1shm.html#a6c8474d5b7dda220ed1f6693fc2e3c89" title="Alias that, given an Arena type (with Arena::construct&lt;T&gt;() which allocates/constructs a T),...">Arena_to_borrower_allocator_arena_t</a>.  <a href="structipc_1_1shm_1_1Arena__to__borrower__allocator__arena.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">struct &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="structipc_1_1shm_1_1Arena__to__borrower__allocator__arena_3_01classic_1_1Pool__arena_01_4.html">Arena_to_borrower_allocator_arena&lt; classic::Pool_arena &gt;</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight">Implementation of <a class="el" href="namespaceipc_1_1shm.html#a6c8474d5b7dda220ed1f6693fc2e3c89" title="Alias that, given an Arena type (with Arena::construct&lt;T&gt;() which allocates/constructs a T),...">Arena_to_borrower_allocator_arena_t</a> for SHM-classic arenas.  <a href="structipc_1_1shm_1_1Arena__to__borrower__allocator__arena_3_01classic_1_1Pool__arena_01_4.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table><table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a id="typedef-members" name="typedef-members"></a>
Typedefs</h2></td></tr>
<tr class="memitem:a6c8474d5b7dda220ed1f6693fc2e3c89"><td class="memTemplParams" colspan="2">template&lt;typename Arena &gt; </td></tr>
<tr class="memitem:a6c8474d5b7dda220ed1f6693fc2e3c89"><td class="memTemplItemLeft" align="right" valign="top">using&#160;</td><td class="memTemplItemRight" valign="bottom"><a class="el" href="namespaceipc_1_1shm.html#a6c8474d5b7dda220ed1f6693fc2e3c89">Arena_to_borrower_allocator_arena_t</a> = typename <a class="el" href="structipc_1_1shm_1_1Arena__to__borrower__allocator__arena.html">Arena_to_borrower_allocator_arena</a>&lt; Arena &gt;::Type</td></tr>
<tr class="memdesc:a6c8474d5b7dda220ed1f6693fc2e3c89"><td class="mdescLeft">&#160;</td><td class="mdescRight">Alias that, given an <code>Arena</code> type (with <code>Arena::construct&lt;T&gt;()</code> which allocates/constructs a <code>T</code>), yields a <code>Borrower_allocator_arena</code> type which can be used as the <code>Arena</code> arg to <a class="el" href="classipc_1_1shm_1_1stl_1_1Stateless__allocator.html" title="Stateless allocator usable with STL-compliant containers to store (or merely read) them directly in S...">stl::Stateless_allocator</a> for the borrower-side counterpart <code>T</code>, usable in <code>Shm_session::borrow_object&lt;T&gt;()</code> to recover so-constructed objects.  <a href="namespaceipc_1_1shm.html#a6c8474d5b7dda220ed1f6693fc2e3c89">More...</a><br /></td></tr>
<tr class="separator:a6c8474d5b7dda220ed1f6693fc2e3c89"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table>
<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
<div class="textblock"><p >Modules for SHared Memory (SHM) support. </p>
<p >At a high level <a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a> is a collection of sub-modules, each known as a <em>SHM-provider</em>, as of this writing most prominently <a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">ipc::shm::classic</a> and ipc::shm::arena_lend::jemalloc; plus provider-agnostic support for SHM-stored native-C++ STL-compliant data structures. See the doc headers for each sub-namespace for further information.</p>
<p >That said here's an overview.</p>
<h3>SHM-providers (<a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">ipc::shm::classic</a>, ipc::shm::arena_lend)</h3>
<p >Generally speaking there are two approaches to making use of <a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a>:</p><ol type="1">
<li>directly; or</li>
<li>via <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a>.</li>
</ol>
<p >While there are applications for approach 1, approach 2 is best by default. It makes the setup of a SHM environment far, far easier &ndash; it is essentially done for you, with many painful details such as naming and cleanup (whether after graceful exit or otherwise) taken care of without your input. Furthermore it standardizes APIs in such a way as to make it possible to swap between the available SHM-providers without changing code. (There are some exceptions to this; they are well explained in docs.)</p>
<p >Regardless of approach, you will need to choose which SHM-provider to use for your application. I (ygoldfel) would informally recommend looking at it from the angle of approach 2, as the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm might clarify the high-level differences between the SHM-providers. I will now <em>briefly</em> describe and contrast the SHM-providers.</p>
<p >As of this writing there are two known types of SHM-providers: <em>arena-sharing</em> and <em>arena-lending</em>, of which only the latter is formalized in terms of its general properties.</p><ul>
<li>Arena-sharing: We have just one such SHM-provider available as of this writing: <a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">shm::classic</a> (SHM-classic; boost.ipc-SHM; boost.interprocess-SHM). In brief: it's built around <a class="el" href="classipc_1_1shm_1_1classic_1_1Pool__arena.html" title="A SHM-classic interface around a single SHM pool with allocation-algorithm services by boost....">shm::classic::Pool_arena</a>, a single-pool thin wrapper around an OS SHM object (pool) handle with a boost.ipc-supplied simple memory allocation algorithm. Both processes in an IPC conversation (session) share one SHM arena (in this case consisting of 1 pool); both can allocate in that shared arena and cross-deallocate. Internally auto-deallocation is handled via an atomic ref-count stored directly in the same SHM arena as the allocated object. Due to SHM-classic's intentional simplicity, the lend/borrow aspects of it (where side/process 1 of a session <em>lends</em> a SHM-allocated object to side/process 2 which <em>borrows</em> it, thus incrementing the conceptual cross-process ref-count to 2) are properties of the arena-object <code>Pool_arena</code> itself.</li>
<li>Arena-lending: We have one SHM-provider as of this writing: shm::arena_lend::jemalloc; it is an application of the formalized arena-lending-SHM-provider paradigm specifically to the commercial-grade 3rd party open-source <code>malloc()</code> provider (memory manager): <a href="https://jemalloc.net">jemalloc</a>. It could be applied to other memory managers; e.g., tcmalloc. (At the moment we feel jemalloc is easily the most performant and customizable open-source <code>malloc()</code>er around.) Generally the memory-manager-agnostic aspects live in shm::arena_lend; while the SHM-jemalloc-specific ones go into shm::arena_lend::jemalloc. A major aspect of arena-lending SHM-providers is the separation of the the arena from the lend/borrow engine (SHM-session). (Those aspects live in <a class="el" href="namespaceipc_1_1session_1_1shm_1_1arena__lend.html" title="Bundles ipc::session::shm support for the various arena-lend-style SHM-providers, as of this writing ...">session::shm::arena_lend</a> and <a class="el" href="namespaceipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc.html" title="Support for SHM-backed ipc::session sessions and session-servers with the SHM-jemalloc (ipc::shm::are...">session::shm::arena_lend::jemalloc</a>; again, the memory-manager-agnostic and -non-agnostic aspects respectively.) With an arena-lending SHM-provider, <em>each</em> of the two processes in a session creates/maintains its own arena, in which the other side cannot allocate; then via the session object the other side <em>borrows</em> an allocated object which it can at least read (but not deallocate; and by default not write-to). Thus process 1 maintains a Jemalloc-managed arena; process 2 borrows objects from it and reads them; and conversely process 2 maintains a Jemalloc-managed arena; process 1 borrows objects from it and reads them. Hence there are 2 process-local <em>SHM-arenas</em> and 1 <em>SHM-session</em> for bidirectional lending/borrowing.</li>
</ul>
<p ><a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">shm::classic</a> is deliberately minimalistic. As a result it is very fast around setup (which involves, simply, an OS SHM-open operation on each side) and around lend/borrow time (when a process wants to share a SHM-stored datum with another process). The negatives are:</p><ul>
<li>It is not (and would be &ndash; at best &ndash; extremely difficult to become) integrated with a commercial-grade memory manager, with features such as anti-fragmentation and thread-caching; hence the allocation/deallocation of objects may be slower compared to heap-based over time. We rely on boost.ipcs's algorithm which lacks the maturity of a jemalloc; and while a custom one could surely replace it, it would be challenging to improve-upon without bringing in a 3rd-party product; such products are not usually designed around being placed <em>entirely</em> into shared memory.</li>
<li>Two processes (at least) intensively write to the same memory area; this in the presence of crashing/zombifying/bugs &ndash; where one process goes bad, while others are fine &ndash; increases entropy and complicates recovery. If process X of multiple co-sharing processes goes down or is ill, the entirety of the SHM-stored data in this system is suspect and should probably be freed, all algorithms restarted.</li>
</ul>
<p >There are no particular plans to make <a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">shm::classic</a> more sophisticated or to formalize its type ("arena-sharing") to be extensible to more variations. It fulfills its purpose; and in fact it may be suitable for many applications.</p>
<p >In contrast shm::arena_lend is sophisticated. A process creates an <em>arena</em> (or arenas); one can allocate objects in arenas. A real memory manager is in charge of the mechanics of allocation; except when it would normally just <code>mmap()</code> a vaddr space for local heap use, it instead executes our internal hooks that <code>mmap()</code> to a SHM-pool; SHM-pools are created and destroyed as needed.</p>
<p >The other process might do the same. It, thus, maintains its own memory manager, for allocations in SHM invoked from that process. Without further action, the two managers and the arenas they maintain are independent and only touched by their respective processes.</p>
<p >To be useful for IPC one would share the objects between the processes. To be able to do so, during setup each process establishes a <em>SHM-session</em> to the other process (multiple sessions if talking to multiple processes). Then one registers each local arena with a SHM-session; this means objects from that arena can be sent to the process on the opposing side of the SHM-session. From that point on, any object constructed in a given arena can be <em>lent</em> (sent) to any process on the opposing side of a session with which that given arena has been registered. This negates the negatives of SHM-classic:</p><ul>
<li>A commercial-grade memory manager efficiently manages the in-SHM heap. If you trust the performance of your regular <code>malloc()</code>/<code>new</code>/etc., then you can trust this equally.</li>
<li>Process 1 does not write to (or at least does not allocate in) a pool managed by process 2; and vice versa. Hence if process X goes down or is ill, the arenas created by the other processes in the system can continue safely. The negatives are a large increase in complexity and novelty; and possible risks of sporadically increased latency when SHM-allocating (as, internally, SHM-pool collections must be synchronized across session-connected processes) and during setup (as, during the initial arena-lend one may need to communicate a large built-up SHM-pool collection). Just to set up a session, one must provide an <a class="el" href="classipc_1_1transport_1_1struc_1_1Channel.html" title="Owning and wrapping a pre-connected transport::Channel peer (an endpoint of an established channel ov...">ipc::transport::struc::Channel</a> for the SHM-session's internal use to synchronize pool collections and more.</li>
</ul>
<p >Lastly, as it stands, the arena-lending paradigm does lack one capability of SHM-classic; it is fairly advanced and may or may not come up as an actual problem:</p>
<p >Imagine long-lived application A and relatively short-lived application B, with (say) serially appearing/ending processes B1, B2, B3 in chronological order. A can allocate and fill an object X1 while B1 is alive; it will persist even after B1 dies and through B2 and B3; B1 through B3 can all read it. But can B1 <em>itself</em> do so?</p><ul>
<li>With <a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">shm::classic</a>: Yes. It's all one arena shared by everyone, readable and writable and allocatable by all.</li>
<li>With shm::arena_lend: No. Anything B1 allocates, by definition, must disappear once B1 exits. The entire arena disappears by the time B2 appears. B2 can read anything that <em>A</em> allocated including before B2 was born, because A is alive as is the arena it maintains; but B1 &ndash; no. In the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm this type of data is known as <em>app-scope</em> in contrast to most data which are <em>session-scope</em>. For data relevant only to each conversation A-B1, A-B2, A-B3, there is no asymmetry: Internally there are 2 arenas in each of the 3 sessions, but conceptually it might as well be a 1 common arena, since both sides have symmetrical capabilities (allocate, read/write, lend; borrow, read). So for session-scope data <a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">shm::classic</a> and shm::arena_lend are identical.</li>
</ul>
<h3>STL support</h3>
<p >The other major sub-module, as mentioned, is agnostic to the specific SHM-provider. It allows one to store complex native C++ data directly in SHM. Namely, arbitrary combinations of STL-compliant containers, <code>struct</code>s, fixed-length arrays, scalars, and even pointers are supported. Both SHM-providers above (<a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">shm::classic</a> and shm::arena_lend::jemalloc) provide the semantics required to correctly plug-in to this system. See doc header for namespace <a class="el" href="namespaceipc_1_1shm_1_1stl.html" title="ipc::shm sub-module providing integration between STL-compliant components (including containers) and...">shm::stl</a> to continue exploring this topic. </p>
</div><h2 class="groupheader">Typedef Documentation</h2>
<a id="a6c8474d5b7dda220ed1f6693fc2e3c89" name="a6c8474d5b7dda220ed1f6693fc2e3c89"></a>
<h2 class="memtitle"><span class="permalink"><a href="#a6c8474d5b7dda220ed1f6693fc2e3c89">&#9670;&nbsp;</a></span>Arena_to_borrower_allocator_arena_t</h2>

<div class="memitem">
<div class="memproto">
<div class="memtemplate">
template&lt;typename Arena &gt; </div>
      <table class="memname">
        <tr>
          <td class="memname">using <a class="el" href="namespaceipc_1_1shm.html#a6c8474d5b7dda220ed1f6693fc2e3c89">ipc::shm::Arena_to_borrower_allocator_arena_t</a> = typedef typename <a class="el" href="structipc_1_1shm_1_1Arena__to__borrower__allocator__arena.html">Arena_to_borrower_allocator_arena</a>&lt;Arena&gt;::Type</td>
        </tr>
      </table>
</div><div class="memdoc">

<p>Alias that, given an <code>Arena</code> type (with <code>Arena::construct&lt;T&gt;()</code> which allocates/constructs a <code>T</code>), yields a <code>Borrower_allocator_arena</code> type which can be used as the <code>Arena</code> arg to <a class="el" href="classipc_1_1shm_1_1stl_1_1Stateless__allocator.html" title="Stateless allocator usable with STL-compliant containers to store (or merely read) them directly in S...">stl::Stateless_allocator</a> for the borrower-side counterpart <code>T</code>, usable in <code>Shm_session::borrow_object&lt;T&gt;()</code> to recover so-constructed objects. </p>
<p >Informally, for informational convenience:</p><ul>
<li>Arena-sharing SHM-providers (<a class="el" href="classipc_1_1shm_1_1classic_1_1Pool__arena.html" title="A SHM-classic interface around a single SHM pool with allocation-algorithm services by boost....">classic::Pool_arena</a> as of this writing), by definition, are symmetric, where each side can both lend and borrow, allocate and write within the same <code>Arena</code>. Hence they will map <code>Arena</code> to itself.</li>
<li>Arena-lending SHM-providers (arena_lend::jemalloc as of this writing), by definition, are asymmetric; the borrowing side can only read, not allocate. Hence they will map <code>Arena</code> to a different borrower-arena type only whose <code>Pointer</code> <em>type</em> member shall be used.</li>
</ul>
<dl class="tparams"><dt>Template Parameters</dt><dd>
  <table class="tparams">
    <tr><td class="paramname">Arena</td><td>SHM arena type that has method of the form <code>shared_ptr&lt;T&gt; construct&lt;T&gt;(...)</code>. </td></tr>
  </table>
  </dd>
</dl>

<p class="definition">Definition at line <a class="el" href="shm_2shm_8hpp_source.html#l00057">57</a> of file <a class="el" href="shm_2shm_8hpp_source.html">shm.hpp</a>.</p>

</div>
</div>
</div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Thu May 2 2024 23:56:39 for Flow-IPC by&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.4
</small></address>
</body>
</html>
